Media foundation media processor

ABSTRACT

A system and method for a media processor separates the functions of topology creation and maintenance from the functions of processing data through a topology. The system includes a control layer including a topology generating element to generate a topology describing a set of input multimedia streams, one or more sources for the input multimedia streams, a sequence of operations to perform on the multimedia data, and a set of output multimedia streams, and a media processor to govern the passing of the multimedia data as described in the topology and govern the performance of the sequence of multimedia operations on the multimedia data to create the set of output multimedia streams. The core layer includes the input media streams, the sources for the input multimedia streams, one or more transforms to operate on the multimedia data, stream sinks, and media sinks to provide the set of output multimedia streams.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of, and claims the benefit of, U.S.patent application Ser. No. 12/025,719 that was filed on Feb. 4, 2008and that is a divisional of, and claims the benefit of, U.S. Pat. No.7,555,540 that was filed on Jun. 25, 2003, each of which is incorporatedherein by reference in its entirety.

TECHNICAL FIELD

This invention relates generally to computing and, more particularly,relates to handling multimedia data in a computing environment.

BACKGROUND OF THE INVENTION

As the abilities of computers expand into entertainment genres that oncerequired separate electronic components, increased efficiency anduser-friendliness is desirable. One solution is Microsoft's®DirectShow®, which provides playback of multimedia streams from localfiles or Internet servers, capture of multimedia streams from devices,and format conversion of multimedia streams. DirectShow® enablesplayback of video and audio content of file types such as Windows MediaAudio (WMA), Windows Media Video (WMV), Moving Pictures Experts Group(MPEG), Apple® QuickTime®, Audio-Video Interleaved (AVI), and WaveformAudio File (WAV). DirectShow® includes a system of pluggable filtercomponents. Filters are objects that support DirectShow® interfaces andcan operate on streams of data by reading, copying, modifying andwriting data to a file. The basic types of filters include a sourcefilter, which takes the data from some source, such as a file on disk, asatellite feed, an Internet server, or a Video Cassette Recorder (VCR),and introduces it into the filter graph which is a connection offilters. The filter graph provides transform filter, which converts theformat of the data, a sync and source filter which receives data andtransmits the data; and a rendering filter, which renders the data, suchas rendering the data to a display device. The data could also berendered to any location that accepts media. Other types of filtersincluded in DirectShow® include effect filters, which add effectswithout changing the data type, and parser filters, which understand theformat of the source data and know how to read the correct bytes, createtimes stamps, and perform seeks.

Therefore, all data passes from filter to filter along with a good dealof control information. When filters are connected using the pins, afilter graph is created. To control the data flow and connections in afilter graph, DirectShow® includes a filter graph manager. The filtergraph manager assists in assuring that filters are connected in theproper order, but the data and much of the control do not pass throughthe filter graph manager. Filters must be linked appropriately. Forexample, the filter graph manager must search for a renderingconfiguration, determine the types of filters available, link thefilters in the appropriate order for a given data type and provide anappropriate rendering filter.

White filters allowed a great deal of reuse of programs, the use offilters also created some unanticipated problems. One of the problemscreated by filters is the large number of Application ProgrammingInterfaces (API's) for the filters which came into being. Each filteressentially has a separate API. Therefore, a given filter must becapable of interfacing to the API for every filter to which it mightattach. Also, the use of filters creates the problem of shutting donnagiven filter problematic. When a given filter in a graph is shut down,any filter that interfaces with the shut down filter requires adifferent associated interface. In general, programming a filter togracefully handle the toss of an interface is difficult, as the state ofthe filter can be unknown when the interface is lost. The loss ofinterfaces, therefore, tends to lead to unpredicted behavior in thefilters and ultimately to ill behaved programs. Further, the overallcontrol in DirectShow® is distributed between two blocks. The interfacebetween the filters controls the data flow while the filter managercontrols the instantiation and removal of filters. Distributing thecontrol in this manner makes software design cumbersome as there areinevitably some control functions which cross the boundary between theblocks. Another problem with DirectShow is that the filters shoulder theresponsibility of media format negotiation and buffer managementfunctionality. Filters communicate with other filters to accomplish thistask. The dependency on filters causes applications building onDirectShow susceptible to bugs and inefficiencies that could beprogrammed into a filter. Thus, a badly written filter could easilybring down the filter graph and an application associated with thefilter graph.

There is a need to address the problems with the DirectShow®architecture. More particularly, there is a need to improve control ofprocessing of multimedia data and address the dependency on filters forcommunications among multimedia components.

SUMMARY OF THE INVENTION

Accordingly, systems and methods of processing multimedia data separatecontrol functions and from data handling functions, thereby providingefficient processing of multimedia streams. A method provides forcreating a topology of connections between one or more multimediacomponents in a topology generating element, the topology describing aset of input multimedia streams, one or more sources for the inputmultimedia streams, a sequence of operations to perform on themultimedia data, and a set of output multimedia streams. The methodfurther provides for transmitting the topology to a media processor, andpassing data according to the topology, the passing governed by themedia processor. The topology generating element, which can be atopology loader or an application, performs outside the scope governedby the media processor. The media processor governs performing thesequence of multimedia operations on the multimedia data to create theset of output multimedia streams. In one embodiment, the multimediacomponents are software objects.

Another embodiment provides a method for changing a first topology inuse by a media processor while the media processor is active. Accordingto the method, the media processor preserves the present state of themedia processor, receives instructions to convert the first topologyinto a second topology, and updates the first topology to the secondtopology in accordance with the instructions. The instructions cancontain the difference between the first topology and the secondtopology. After updating the first topology to the second topology, themedia processor resumes the interface activity after updating the firsttopology to the second topology, sends messages to an application.Before the topology changes the media processor can be configured toallow message calls. The instructions to convert the first topology canbe received via a message from an external source to initiate theprocess of changing the first topology.

Another embodiment is directed to a method of determining how to use aset of multimedia components to perform a sequence of multimediaoperations on one or more streams of multimedia data. The method isrecursive in that the use of the multimedia components is determined byquerying prior components for available sample data. The method includeslocating one or more multimedia components with outputs connected to aninput of a sink device, querying the multimedia components to determineif a sample is available, the querying can include checking inputs tothe multimedia components if a sample is not available. If the inputs donot have a sample available, checking a media source feeding themultimedia components for a sample. If the media source does not have asample available, the method provides for performing an end of filefunction or declaring an error condition. If a sample is available, themethod provides for moving the sample to a next multimedia component ofthe multimedia components.

Another embodiment is directed to a method for retrieving a section of amedia stream, which can be referred to as “scrubbing.” The methodincludes caching the section of a media stream. The cached section ofthe media stream can contain a presentation point of the media stream.The method then provides for receiving a request from an external sourceto the media processor to retrieve the cached section of the mediastream, and searching to identify whether the sec on of the mea streamwas cached. If the section of the media stream was cached, the methodprovides for transferring the requested cached section. The caching canbe according to user settings in an application, which can includewhether or not to cache, a number of frames and a number of samples tobe contained in the cached section. In one embodiment, the cachedsection is continuous video data.

BRIEF DESCRIPTION OF THE DRAWINGS

While the appended claims set forth the features of the presentinvention with particularity, the invention, together with its objectsand advantages, may be best understood from the following detaileddescription taken in conjunction with the accompanying drawings ofwhich:

FIG. 1 is a block diagram generally illustrating an exemplarydistributed computing system with which the present invention can beimplemented;

FIG. 2 is a block of a media foundation system in accordance withembodiments of the present invention.

FIG. 3 is a flow chart of an example of data flow in the media enginerequired to play a DVD in accordance with embodiments of the presentinvention.

FIG. 4 is a block diagram illustrating how data flow is implemented inthe media processor in accordance with embodiments of the presentinvention.

FIG. 5 is a flowchart of a dynamic topology change in the mediaprocessor in accordance with embodiments of the present invention.

FIG. 6 is a flowchart exemplifying a process of scrubbing in accordancewith embodiments of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Turning to FIG. 1, an exemplary computing device 100 on which theinvention may be implemented is shown. The computing device 100 is onlyone example of a suitable computing device and is not intended tosuggest any limitation as to the scope of use or functionality of theinvention. For example, the exemplary computing, device 100 is notequivalent to any of the computing devices 10-17 illustrated in FIG. 1.The exemplary computing device 100 can implement one or more of thecomputing devices 10-17, such as through memory partitions, virtualmachines, or similar programming techniques, allowing one physicalcomputing structure to perform the actions described below as attributedto multiple structures.

The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Indistributed computing environments, tasks can be performed by remoteprocessing devices that are linked through a communications network. Ina distributed computing environment, program modules may be located inboth local and remote computer storage media including memory storagedevices.

Components of computer device 100 may include, but are not limited to, aprocessing unit 120, a system memory 130, and a system bus 121 thatcouples various system components including the system memory to theprocessing unit 120. The system bus 121 may be any of several types ofbus structures including a memory bus or memory controller, a peripheralbus, and a local bus using any of a variety of bus architectures. By wayof example, and not limitation, such architectures include IndustryStandard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus,Enhanced ISA (EISA) bus, Video Electronics Standards Associate (VESA)local bus, and Peripheral Component interconnect (PCI) bus also known asMezzanine bus.

Computing device 100 typically includes a variety of computer readablemedia. Computer readable media can be any available media that can beaccessed by computing device 100 and includes both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer readable media may comprise computerstorage media and communication media. Computer storage media includesboth volatile and nonvolatile, removable and non-removable mediaimplemented in any method or technology for storage of information suchas computer readable instructions, data structures, program modules orother data. Computer storage media includes, but is not limited to,Random Access Memory (RAM), Read-Only Memory (ROM), ElectricallyErasable Programmable ROM (EEPROM), flash memory or other memorytechnology, Compact Disc ROM (CD-ROM), digital versatile disks (DVD) orother optical disk storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices, or any other mediumwhich can be used to store the desired information and which can beaccessed by computing device 100. Communication media typically embodiescomputer readable instructions, data structures, program modules orother data in a modulated data signal such as a carrier wave or othertransport mechanism and includes any information delivery media. Theterm “modulated data signal” means a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationmedia includes wired media such as a wired network or direct-wiredconnection, and wireless media such as acoustic, Radio Frequency (RF),infrared and other wireless media. Combinations of the any of the aboveshould also be included within the scope of computer readable media.

The system memory 130 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 131and random access memory (RAM) 132. A basic input/output system 133(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 110, such as during start-up, istypically stored in ROM 131. RAM 132 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 120. By way of example, and notlimitation, FIG. 1 illustrates operating system 134, applicationprograms 135, other program modules 136, and program data 137.

The computing device 100 may also include other removable/non-removable,volatile/nonvolatile computer storage media. By way of example only,FIG. 1 illustrates a hard disk drive 141 that reads from or writes tonon-removable, nonvolatile magnetic media, a magnetic disk drive 151that reads from or writes to a removable, nonvolatile magnetic disk 152,and an optical disk drive 155 that reads from or writes to a removable,nonvolatile optical disk 156 such as a CD ROM or other optical media.Other removable/non-removable, volatile/nonvolatile computer storagemedia that can be used in the exemplary operating environment include,but are not limited to, magnetic tape cassettes, flash memory cards,digital versatile disks, digital video tape, solid state RAM, solidstate RUM, and the like. The hard disk drive 141 is typically connectedto the system bus 121 through a non-removable memory interface such asinterface 140, and magnetic disk drive 151 and optical disk drive 155are typically connected to the system bus 121 by a removable memoryinterface, such as interface 150. The term “computer storage media” asused herein refers specifically to a statutory article(s) of manufacturethat is not a signal or carrier wave per se.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 1, provide storage of computer readableinstructions, data structures, program modules and other data for thecomputing device 100. In FIG. 1, for example, hard disk drive 141 isillustrated as storing operating system 144, application programs 145,other program modules 146, and program data 147. Note that thesecomponents can either be the same as or different from operating system134, application programs 135, other program modules 136, and programdata 137. Operating system 144, application programs 145, other programmodules 146, and program data 147 are given different numbers heretoillustrate that, at a minimum, they are different copies. A user mayenter commands and information into the computing device 100 throughinput devices such as a keyboard 162 and pointing device 161, commonlyreferred to as a mouse, trackball or touch pad. Other input devices (notshown) may include a microphone, joystick, game pad, satellite dish,scanner, or the like. These and other input devices are often connectedto the processing unit 120 through a user input interface 160 that iscoupled to the system bus, but may be connected by other interface andbus structures, such as a parallel port, game port or a universal serialbus (USB). A monitor 191 or other type of display device is alsoconnected to the system bus 121 via an interface, such as a videointerface 190. In addition to the monitor, computers may also includeother peripheral output devices such as speakers 197 and printer 196,which may be connected through a output peripheral interface 195.

The computing device 100 operates in a networked environment, such asthat shown in FIG. 1, using logical connections to one or more remotecomputers. FIG. 1 illustrates a general network connection 171 to aremote computing device 180. The general network connection 171 can beany of various different types of network connections, including a LocalArea Network (LAN), a Wide-Area Network (WAN), networks conforming tothe Ethernet protocol, the Token-Ring protocol, or other logical orphysical networks such as the Internet or the World Wide Web.

When used in a networking environment, the computing device 100 isconnected to the general network connection 171 through a networkinterface or adapter 170, which can be a network interface card, amodem, or similar networking device. In a networked environment, programmodules depicted relative to the computing device 100, or portionsthereof, may be stored in the remote memory storage device. Thoseskilled in the art will appreciate that the network connections shownare exemplary and other means of establishing a communications linkbetween the computers may be used.

In the description that follows, the invention will be described withreference to acts and symbolic representations of operations that areperformed by one or more computing devices, unless indicated otherwise.As such, it will be understood that such acts and operations, which areat times referred to as being computer-executed, include themanipulation by the processing unit of the computing device ofelectrical signals representing data in a structured form. Thismanipulation transforms the data or maintains it at locations in thememory system of the computing device, which reconfigures or otherwisealters the operation of the computing device in a manner well understoodby those skilled in the art. The data structures where data ismaintained are physical locations of the memory that have particularproperties defined by the format of the data. However, while theinvention is being described in the foregoing context, it is not meantto be limiting as those of skilled in the art will appreciate thatseveral of the acts and operation described hereinafter may also beimplemented in hardware.

Referring to FIG. 2, a block diagram illustrates a media foundationsystem 200. Core layer 211 includes media source 210, transforms 208,and media sink 230. Media foundation system 200 is shown coupled toapplication 202 to receive and send media streams. Control layer 201includes media engine 260, media session 240, media processor 220 andtopology loader 250. Data flows through the media foundation 200 bybeginning with a media source 210 into media session 240 and then intomedia processor 220. From media processor 220 the data will flow intotransforms 208 and back to media processor 220 one or more times. Thedata will then flow from media processor 220 into media session 240 andthen into stream sinks 212. Media engine 260 provides control to aninterface to the application 202 and provides overall control of controllayer 201, and the topology loader 250 ensures that events prescribed ina topology occur. The media foundation system 200 provides interfacesand a layout for connecting streaming media objects.

The core layer includes media source component 210, and media sinkcomponent 230. Also included are stream sources 214 which operate underthe control of media source 210; and stream sinks 212 which operateunder the control of media sink 230. Stream sources 214 transfermultimedia data from storage or capture devices to control layer 201 andstream sinks 212 transfer multimedia data from media engine 260 torendering or storage devices (not show). Media source component 210implements state machines which provide control of stream sources 214.Media sink component 230 implements state machines which provide controlof stream sinks 212. In each case, the state processing and datamovement are separated.

Media source 210, media sink 230 and transforms 208, together withstream sources 214 and stream sinks 212 include objects that make uppart of core layer 211. These components are programmatic objects whichimplement a predefined function. Media source 210 and stream sources 214provide either capture or retrieval of multimedia data and provide thisdata to media session 240. The sources of data include but are notlimited to a disk such as a hard drive, CD, or DVD, the internet, randomaccess memory (RAM), video RAM, video cameras, scanners, still imagecameras, and microphones. Media sink 230 includes objects which controlthe transfer of data in stream sinks 212. Stream sinks 212 consist ofobjects which accept data from control layer 201 for storage orrendering. Sinks of data include but are not limited to a disk such as ahard drive, writable CD, or writable DVD, a broadcast over a computernetwork, such as the Internet, printers, display devices such asmonitors, and speakers. The data for both the media source 210 and mediasink 230 can be transported over many mediums including but not limitedto Ethernet, wireless networks, analog cables before digitization, USB,IEEE 1384, parallel port, serial port, and disk interfaces.

Transforms 208 include objects which manipulate the data. Thesetransforms can include encoders, decoders, splitters, multiplexers,audio processing such as bass and treble control for adding effects suchas reverb, video processing such as adjusting color masks, imagesharpening, and contrast or brightness control. The encoders anddecoders handle both audio, video, and image data. Video data types caninclude MPEG, Apple Quicktime®, AVI, and H.263 and Windows Media Video(WMV). Note that many of the video standards are true multimediastandards in that these standards have provisions to transfer both audioand video. Image data formats include JPEG, GIF, Fax, and Tiff. Audiostandards can include MP3, PCM, ADPCM, as well as standards for CDplayback and Windows Media Audio (WMA). Transforms 208 can be usedconvert data from one format to another. For example, a transform 208can convert a JPEG image into a format suitable for display on amonitor.

Turning to FIG. 3, a flow chart of a typical multimedia process is show.Assume a user wishes to view a DVD. An application will be launched toallow the user to view a DVD. The application has a graphical userinterface (GUI) allowing the user to perform such functions as play,stop, pause, fast forward, and rewind. In block 302, the user selectsthe play button and the application sends a message to the media enginecomponent of media foundation. The message contains the information thatthe application wishes to view a DVD. In block 304, the media enginesends messages to media session and the topology loader telling theseblocks to start playing a DVD. In block 306, the topology loader sets upthe topology. The topology provides a path that the data streams takethrough the media and stream sources, the transforms, and the media andstream sinks. The topology loader will pass this topology on to themedia processor. The media processor sets up and implements thetopology. In block 308, the media processor will send messages to thecore layer components to instantiate the objects called out by thetopology loader. In addition to calling the core layer objects in theproper order and passing data between the objects, the data rate iscontrolled such that the audio and video are synchronized and renderedat the desired rate. The data rate can be determined in the mediasession. In block 310, the media session will query each object todetermine that the desired rate can be supported and pass a message tothe media processor with the rate information. In block 312, the mediaprocessor determines the clock rate of a rate determining object in thecore level, usually a media sink, and sets this clock rate. In block314, the media processor then calls the core level and passes databetween objects as required by the topology. The data is ultimatelyrendered to the speakers and monitor by media sinks.

The media processor is the object that performs the data flow asdescribed by a topology. The media processor is initialized with atopology describing the data flow, and exposes itself to the user viathe media source interface. Thus, once configured, the media processoractually looks like a media source to the user. Media processor exposesa number of media streams based on the topology. There will be one mediastream for each output node in the topology.

FIG. 4 shows a topology. The data flow is driven by user calls to fetcha sample on one of the media processor's media streams. Consider theupper path in FIG. 4. The data flow works by starting with an outputobject in the topology (sink object 412) and walking recursively througha list of objects which generate data. The process in the upper path ofFIG. 4 starts with sink object 412. The only output node connected toinput 416 is output 418 connected to transform object 408. If transformobject 408 has a sample available, then media processor 220 reads thesample and writes the sample to sink object 412. If transform object 408does not have a sample available, then media processor 220 looks at theinput of transform object 408, which is shown as 420, which is connectedto output 422 for transform object 404. Transform object 404 is thenqueried for a sample. If a sample is available, the sample is read bymedia processor 220 and written to transform object 408. If no sample isavailable, then media processor 220 again moves one node to the left,and queries media stream object 403 for an available sample. Mediastream object 403 are loaded via source object 402. If a sample isavailable from media stream object 403, the sample is retrieved frommedia stream object 403 to transform object 404. If no sample is found,then media processor will request that media stream object 403 read asource. Any time that a valid sample is found and passed to the nextblock, the process starts over. The process is completed by operatingwith the objects once the data is located. Transform object 404 operateson the data and passes the transformed data to transform object 408 tooperate on the data and then passes the transformed data to sink object412 to complete the process of generating a sample at the output. In oneembodiment, the media processor keeps a list objects that have inputs toother objects. With this list, the media processor can look at theoutput object and determine from which object the media processor needsto retrieve media data.

In one embodiment, the list tracks the type of node holding the objectsand other information about the objects. Each type of object has adifferent manner of generating data at the media processor's request.Transform nodes contain transform objects, which have a set of callsthat are used to provide input media samples and generate output mediasamples. Tee nodes provide an indication for the media processor to copysamples as needed. Source nodes have a media stream, which provides anasynchronous call by which the media processor asks the stream for data,and the stream later provides the data when the data is available.

Embodiments are directed to a data flow using media processor 220. In anembodiment, the data flow is asynchronous. That is, a user makes a callto generate data for a given media stream, and media processor 220 thengenerates the data, and notifies the user when the data is available. Inone embodiment, components referenced by a topology do not make calls toeach other. Rather, media processor 220 is responsible for allcommunication. By having a centrally located communication method dataflow is consistent, and there is greater interoperability betweencomponents.

In one embodiment, media processor 220 does not address each aspect ofthe data flow. For example, in one embodiment, media processor 220 hasno control over media samples.

In terms of the overall media foundation architecture, the mediaprocessor is commonly used in the context of the media engine. Mediaengine 260 is the component that an application 202 uses directly whenusing Media Foundation architecture shown in FIG. 2. Media engine 260can be configured by the application 202 specifying the source of data(generally a filename, URL, or device, or a complex configuration thatspecifies multiple simple sources), and the destination of the data(such as an output multimedia file, or a rendering device like a videocard). Media engine 260 is then controlled at runtime through suchcommands as Start, Stop, etc. Thus, media engine 260 uses the otherMedia Foundation components to accomplish this functionality, and is themain object that an application 202 uses to perform multimediaoperations. Media processor 220 can be controlled directly by a mediaprocessing session, and used in conjunction with topology loader 250.The media processing session is the object media engine 260 uses tooperate media sources 210, media processor 220, media sinks 230, andtopology loader 250.

In one embodiment, media processor 220 supports tee objects andtransform objects with more than one input or output. For example, atransition is generally implemented as a transform object, and thetransform object would have two inputs and one output.

In the case in which a node has more than one input, the media processor220 performs the following method: when trying to generate input datafor the transform, media processor 220 selects just one of the inputsbased on the timestamps of the previous media samples and generates datafor that input. Every time media processor 220 provides an input sampleto a transform object, media processor 220 attempt to generate outputdata for the transform. If the transform does not generate any data,media processor 220 provides an input sample to the transform object,possibly the same input that was already used.

In the case where a node has more than one output, media processor 220needs more information about the outputs. The topology loader willindicate the properties of the output, which can be either primary ordiscardable.

The primary output is used as the primary memory allocator for samplespassed to the input nodes. Discardable outputs are not guaranteed to getall samples that go into the node; if the user hasn't requested a samplefor that output when an input sample has been generated, then thediscardable output will simply not receive the sample. The discardableconcept is useful in scenarios wherein it is preferable to lose datarather than slow processing down, such as a preview display in anencoder application.

An important capability of a multimedia system is the ability to changethe properties of the multimedia session white running. For instance, aplayback application might switch from playback in windowed mode intofull screen mode. Or an encoding application might switch from encodingone camera to another camera. The disclosed media foundation systemhandles these capabilities via a change in the topology.

However, in an embodiment, media processor 220 never changes thetopology itself; topology changes are always be done by anothercomponent and then communicated to the media processor. If mediaprocessor 220 is being controlled by media session 240, media session240 is responsible for using the topology loader to create fulltopologies for media processor 220 and then communicating these fulltopologies to the media processor. However, in one embodiment any directuser of the media processor can also perform topology changes on themedia processor.

In one embodiment topology changes can be static or dynamic. A statictopology change takes place when media processor 220 is not processingdata and represents a full replacement of the old topology with the newtopology. A dynamic topology change takes place when media processor isrunning and also may change only part of the topology while maintainingother parts of the topology intact.

There are several mechanisms through which a dynamic topology change canbe generated. The first is media source generated. In this case, one ofthe media sources internal to media processor 220 detects that itsformat has changed in some manner; a media stream has changed, a mediastream has ended, or a new media stream has been created. The mediasource notifies media processor 220 that the change has occurred; mediaprocessor 220 forwards this notification on to the user to process, andstops processing, data. The user is responsible for creating the newtopology and sending it on to the media processor. The second istopology generated. In this case, the topology itself containsinformation that it will change at some point; one or more of thetopology objects has an expiration time set as a property. When mediaprocessor 220 detects that the expiration time has been reached, itnotifies the user, and stops processing data. The final type is usergenerated. In this case the user simply sets a new topology on mediaprocessor 220 while media processor 220 is running. In all cases, mediaprocessor 220 reacts to the topology change in the same manner:

Referring to FIG. 5, the steps in a dynamic topology change are shown ina flowchart. In block 502, a topology change is requested of the mediaprocessor. In block 504, media processor 220 first makes anynotifications to the user that are required, for instance that a mediasource has changed formats. In block 506, media processor 220 stopsprocessing and maintains its state such that it can resume processing onany nodes that remain in the new topology. Media processor 220 also putsitself into a state such that any attempt by the user to continuerunning will simply wait until the topology change is complete. In block508, media processor 220 then receives the new topology. The newtopology could come from the topology loader or the application. Inblock 510, media processor 220 updates its internal topology to matchthe new topology. In block 512, media processor 220 informs theapplication that the new topology is in place and resumes processing.

Media processor 220 is designed to support running at arbitrary rates.The media source interface is a pull interface; that is, the user canpull samples as quickly as possible. So from this sense media processor220 automatically supports arbitrary rates. However, in many cases themedia sources or transforms also have optimizations or improved behaviorwhen they know the rate. Thus, when the user tells media processor 220explicitly to run at a given rate, media processor 220 will query anymedia sources and transforms in the topology to determine their ratecapabilities. If the given rate is supported by all media sources andtransforms, media processor 220 then set the rate of all media sourceobjects and transform objects such that the media sources and transformobjects can make appropriate changes to their behavior. Running themultimedia stream in reverse is a special case of a rate change.

Scrubbing is defined as being able to quickly seek within the multimediapresentation and retrieve a small number of media samples at thatposition. Due to the complexity of decoding multimedia data, scrubbingis more complex than seeking into the multimedia presentation and simplytakes too long. This is particularly true for compressed video data,which often has a large time delay between independently decodable keyframes.

To optimize scrubbing, media processor 220 has logic to enable caching acertain amount of pre-decoded samples, generally near the currentposition of the presentation, such that a seek to one of thesepre-decoded samples will allow media processor 220 to generate thedesired sample in a timely manner. This caching behavior is configurableby the application, to allow the tradeoff between memory usage and goodscrubbing performance.

FIG. 6 is a flowchart of the process used to optimize scrubbing. Block602 provides that media processor 220 applies logic to each samplegenerated at a node regarding caching of sample data. Decision block 604provides that when deciding whether to cache sample data, mediaprocessor first determines if the sample data has been cached. If yes,block 606 provides for doing nothing. If no, block 608 provides formedia processor 220 to determine caching settings, as set by the user.The caching settings can include an identification of nodes for whichnodes caching is desired, the frequency of caching, the maximum amountof memory to use for caching, and the like. Block 610 provides forapplying the settings to determine to cache the sample or not andcaching as necessary.

Block 612 provides for media processor 220 to receive a data “seek”issued by application 202. A data seek can be a request, for example,for the next frame of data to enable a frame by frame advance of a videostream. Upon receiving the seek, media processor 220, for each node,checks to see if the requested data is present in cache in decisionblock 614. If present, block 616 provides for sending the data back toapplication 202 instead of regenerating the sample data. If not present,block 618 provides for using the appropriate transform, source or otherappropriate component to generate the sample data.

Some video decoders support a mode in which pre-decoded samples can beused to initialize the state of the decoder. In this case, it ispossible to cache only a fraction of the pre-decoded samples and stillmaintain good scrubbing performance. For instance, if every fourthpre-decoded sample is cached and the user seeks to one of the samplesnot in the cache, at most, three samples need be decoded to generate thedesired output sample.

In view of the many possible embodiments to which the principles of thisinvention may be applied, it should be recognized that the embodimentdescribed herein with respect to the drawing figures is meant to beillustrative only and should not be taken as limiting the scope ofinvention. For example, those of skill in the art will recognize thatthe elements of the illustrated embodiment shown in software may beimplemented in hardware and vice versa or that the illustratedembodiment can be modified in arrangement and detail without departingfrom the spirit of the invention. Therefore, the invention as describedherein contemplates all such embodiments as may come within the scope ofthe following claims and equivalents thereof.

We claim:
 1. A method performed on a computing device, the methodcomprising: stopping, by the computing device in response to a topologychange request to change from processing media data in accordance with afirst topology to processing the media data in accordance with a secondtopology, a media processor that is processing media data in accordancewith the first topology; and resuming, by the computing device inresponse to the media processor receiving the second topology, thestopped media processor, where the resumed media processor continues theprocessing the media data in accordance with the received secondtopology.
 2. The method of claim 1 further comprising receiving, inadvance of the stopping, the topology change request.
 3. The method ofclaim 1 where the first topology defines a path of the media datathrough at least a source, a transform, and a sink.
 4. The method ofclaim 1 further comprising informing, by the media processor inconnection with the resuming, an application of a change of topology. 5.The method of claim 1 further comprising maintaining, by the mediaprocessor in connection with the stopping, a state of the mediaprocessor.
 6. The method of claim 5, the maintained state sufficient forresuming any node of the first topology that remains in the secondtopology.
 7. The method of claim 5 further comprising blocking, by thecomputing device, any attempt by a user, between the stopping and theresuming, to continue the processing the media data.
 8. At least onememory storage device storing computer-executable instructions that,based on execution by a computing device, configure the computing deviceto perform actions comprising: stopping, by the computing device inresponse to a topology change request to change from processing mediadata in accordance with a first topology to processing the media data inaccordance with a second topology, a media processor that is processingmedia data in accordance with the first topology; and resuming, by thecomputing device in response to the media processor receiving the secondtopology, the stopped media processor, where the resumed media processorcontinues the processing the media data in accordance with the receivedsecond topology.
 9. The at least one memory storage device of claim 8,the actions further comprising receiving, in advance of the stopping,the topology change request.
 10. The at least one memory storage deviceof claim 8 where the first topology defines a path of the media datathrough at least a source, a transform, and a sink.
 11. The at least onememory storage device of claim 8, the actions further comprisinginforming, by the media processor in connection with the resuming, anapplication of a change of topology.
 12. The at least one memory storagedevice of claim 8, the actions further comprising maintaining, by themedia processor in connection with the stopping, a state of the mediaprocessor.
 13. The at least one memory storage device of claim 12, themaintained state sufficient for resuming any node of the first topologythat remains in the second topology.
 14. The at least one memory storagedevice of claim 12, the actions further comprising blocking, by thecomputing device, any attempt by a user, between the stopping and theresuming, to continue the processing the media data.
 15. A systemcomprising a computing device and at least one program module togetherconfigured for performing actions comprising: stopping, by the computingdevice in response to a topology change request to change fromprocessing media data in accordance with a first topology to processingthe media data in accordance with a second topology, a media processorthat is processing media data in accordance with the first topology; andresuming, by the computing device in response to the media processorreceiving the second topology, the stopped media processor, where theresumed media processor continues the processing the media data inaccordance with the received second topology.
 16. The system of claim15, the actions further comprising receiving, in advance of thestopping, the topology change request.
 17. The system of claim 15 wherethe first topology defines a path of the media data through at least asource, a transform, and a sink.
 18. The system of claim 15, the actionsfurther comprising informing, by the media processor in connection withthe resuming, an application of a change of topology.
 19. The system ofclaim 15, the actions further comprising maintaining, by the mediaprocessor in connection with the stopping, a state of the mediaprocessor sufficient for resuming any node of the first topology thatremains in the second topology.
 20. The system of claim 19, the actionsfurther comprising blocking, by the computing device, any attempt by auser, between the stopping and the resuming, to continue the processingthe media data.